Vehicle simulator having head-up display

ABSTRACT

A vehicle simulator having a projection system for projecting a generated scene and symbology onto a common viewing screen. Tracking apparatus is provided for providing position and/or angular orientation signals representative of the eye of a person viewing the projected scene and symbology. An image generation system is included for generating the scene and symbology for the projection system, such scene and symbology being generated as a function of the position and/or angular orientation signals provided by the tracking apparatus. Such symbology is representative of symbology produced by a head-up display. The image generation system generates the scene and symbology with characteristics which simulate vignetting and/or occulting effect characteristics of the head-up display. The OTW scene and symbology are projected onto a screen and are co-located while the image generator, base or data of the eye position of the user, produces the symbology to simulate the collimation, vignetting and occulting effects of the HUD optical type used in the vehicle that is simulated.

BACKGROUND OF THE INVENTION

[0001] This invention relates generally to vehicle simulators and moreparticularly to vehicle simulators having head-up displays.

[0002] It is known in the art to use vehicle simulators, such asaircraft flight or tank simulators, to train operators of such vehicles.

[0003] With respect especially to aircraft flight simulators, manyaircraft, particularly fighter aircraft, have head-up displays whichenable a pilot to view the outside environment in front of the aircrafttogether with information which is typically displayed on an instrumentpanel of the aircraft. The head-up display enables the pilot to observea scene outside the aircraft (i.e., an “out-the-window” (OTW) scene) andat the same time to see, i.e., in superposition with the OTW scene,information (“symbology”), which the pilot may need, such as altitude,speed, a pointer to a target, etc.

[0004] An example of one such head-up display (HUD) is shown in FIG. 1wherein a head-up display (HUD) optical system 10 projects informationprovided on the HUD's display device, such as a cathode ray tube (CRT)12, to the eye 14 of an observer through an optical system 16. Theoptical system 16 is used to collimate the images (“symbology”), e.g.,alphanumeric characters, lines, target pointers, etc.) produced on theHUD display device 12. The collimated HUD images are then viewed insuperposition with the OTW imagery through the beamsplitter 18.

[0005] The HUD system 10 in the aircraft shown in FIG. 1 is focused atinfinity (i.e., collimated) because the OTW scene being observed by thepilot is also typically at “infinity”. In the HUD system 10 shown inFIG. 1, if the pilot moves his head to the right, for example, not onlydoes the target he is observing move to the right, but the symbology onthe beamsplitter 18, such an example a pointer generated by the HUD 10pointing to a target, also appears at the eye 14 to move to the right onthe beamsplitter 18, thereby remaining on the target.

[0006] Referring to FIG. 2, a flight simulator 19 of the prior art isshown. The simulator 19 has the cockpit portion 21 of the aircraft,including has a HUD system 10. Simulator 19 includes a projector 24driven by an image generator 26 to produce the simulated OTW scene, andthe generated OTW scene is projected by the projector 24 onto a screen28 for observation by the pilot being trained.

[0007] In the simulator, as in the real world, an acceptablesuperposition of the HUD and OTW imagery occurs when the two images arefocused at the same distance. The screen 28, however, is not at infinitybut relatively close to the eye 14 of the pilot, and consequently, theactual vehicle's HUD, which is focused at infinity for use in a realenvironment, cannot be used without modification.

[0008] One approach for making a HUD for such a simulator is to modifythe optical system in the HUD 10 so that it focuses at the same distancefrom the eye 14 as the screen 28, i.e., such that the optical system 16of the HUD 10 makes the symbology appear to the eye 14 as if it werelocated on the screen 28. The HUD optical system 16 must be modified sothat a simulated, distant target being projected on the screen 28, and asymbology pointer (generated by the HUD 10) pointing to the target(which is generated on the OTW screen 28) appear co-located wherever thepilot in the simulator moves his head.

[0009] One problem with this design is that each different type ofaircraft usually has a different HUD type, or multiple HUD types, andsimulators for a given aircraft may use different displays formingimages at differing distances from the user's eye. The optical system 16for the HUD system 10 in the simulator 19 is a function of both thescreen placement and the HUD type used in the simulator 19. A differentoptical system 16 must therefore be designed for each HUD type used witheach different display having a different screen distance from the eye.Further, as refocusing requirements become shorter and shorter, to matchdecreased distances in the simulator's OTW display, it becomes more andmore difficult to refocus the HUD optics while maintaining the actualHUDs FOV, vignetting characteristics, and mechanical packaging.

[0010] A technique to avoid this problem is to project the HUD symbologyon the same screen on which the OTW scene is projected, as shown thesimulator 19′ of FIG. 3. Both the OTW projector 24, which projects theOTW scene, and a HUD projector 30, which projects the HUD symbology,appear on the same screen 28, and consequently, the OTW scene and thesymbology are physically co-located.

[0011] However, referring again to FIG. 1, in an actual cockpitenvironment, the HUD system 10 provides the symbology on only a limitedportion of the field of view available to the pilot. If the pilot moveshis head around in an actual cockpit as in FIG. 1, portions or all ofthe symbology may or may not be visible to the pilot due to opticallimitations of the HUD system, herein described as occulting orvignetting. The term “occulting” is meant to broadly describe anyblocking or interruption in the visibility of the symbology, such as bystructures of the cockpit or in the HUD optical system. The term“vignetting” refers to a type of occulting which is caused by movementof the viewer's eye beyond the optically functional portions of the lenssystem which is normally in the center of the lens or lenses. Theocculting or vignetting is caused by a combination of things in the HUDoptical system 10, i.e., the beamsplitter 18, the lenses and lens framesof the optics, and the CRT. For example, if the pilot's head is moved sothat some of the HUD FOV falls outside of the beamsplitter coverage 18,that portion is no longer viewable by the pilot.

[0012] Consequently, in the arrangement shown in FIG. 3, whileprojection of the HUD generated symbology and the OTW scene onto acommon screen 28 produces the desired co-location effect, the systemdoes not simulate the vignetting or occulting characteristics associatedwith the real HUD being simulated. The prior art therefore fails toprovide a realistic simulation of a vehicle heads up display.

SUMMARY OF THE INVENTION

[0013] It is therefore an object of the present invention to provide animproved head-up simulator that better simulates the real HUD system inan economical way.

[0014] In accordance with the present invention, a vehicle simulator isprovided which includes a projection system for projecting a generatedscene and symbology onto a common viewing screen. Tracking apparatus isprovided for producing position and/or angular orientation signalsrepresentative of the position of the eye of a person viewing theprojected scene and symbology on the common viewing screen. An imagegeneration system is included for generating the scene and symbology forthe projection system, the symbology being generated as a function ofthe position and/or angular orientation signals provided by the trackingapparatus.

[0015] The OTW scene and the symbology are projected onto a commonscreen and thus are co-located, while the image generator, using dataindicative of the position of an eye of the person in the simulator,produces the symbology in a way which simulates the vignetting orocculting effect characteristic of the HUD optical type being simulated.

[0016] Other features of the invention will become more readily apparentfrom the detailed description, and the scope of the invention will bedescribed by the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

[0017]FIG. 1 is a schematic showing a cockpit portion of an aircrafthaving a HUD enabling a pilot to simultaneously view a scene outside thecockpit and symbology produced by the HUD according to the PRIOR ART;

[0018]FIG. 2 is a schematic showing a simulator for training a pilot inuse of the aircraft of FIG. 1 according to the PRIOR ART;

[0019]FIG. 3 is a schematic showing a simulator for training a pilot inuse of the aircraft of FIG. 1 according to the PRIOR ART;

[0020]FIG. 4 is a schematic showing a flight simulator adapted to traina pilot in operation of the aircraft of FIG. 1 according to theinvention; and

[0021]FIG. 5 is a diagram illustrating the use of image generation masksin simulating the vignetting and occulting characteristics of an actualHUD in a symbology image generator such as that shown in the simulatorof FIG. 4;

[0022]FIG. 6 is a diagram of the modeled cross section of the occultingmasks for simulating the occlusion and vignetting effect caused by anon-pupil forming HUD in the simulator of FIG. 4; and

[0023]FIGS. 7 and 8 are cross section diagrams illustrating vignettingusing the diagramed model of FIG. 6.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] Referring to FIG. 4, a vehicle simulator 40, here an aircraftflight simulator, is shown. The system 40 includes a projection system42 for projecting a generated OTW scene (represented by solid arrow 44)and symbology (represented by the dotted arrow 46) onto a common viewingscreen 48. Other methods of presenting images to the user may also beused, including a CRT display, an LCD display, various shaped screenswith varying surfaces, viewed directly or through optical systems, orhelmet-mounted displays. Tracking apparatus 50 is provided to giveposition and/or angular orientation signals representative of theposition of the eye 19 of a person, here a pilot being trained in thesimulator 10, and viewing the projected scene 44 and the symbology 46 onscreen 48 through canopy 52.

[0025] The simulator of the preferred embodiment includes a hostcomputer 53 which receives data from the simulator and processes thedata to continually define the changing simulated situationalenvironment, e.g., the background landscape, targets, and any otherobjects that exist in the simulated situation. The situational data istransmitted to an image generation system 50 for generating the OTWscene 44 and the symbology 46 for the projection system 12. Thesymbology 46 is generated as a function of the position and/or angularorientation signals provided by the tracking apparatus 50. The OTW scene44 and the symbology 46 are projected onto the screen 14 and thus areco-located while the image generator 54, having data representing theposition of at least one eye 19 of the person in the simulator 40,produces the symbology 46 in a way which incorporates the vignetting andocculting effect of the HUD optical type being simulated in thesimulator 40.

[0026] Although potentially a single high resolution projector might beused in projector 42, in the preferred embodiment, the projection system42 includes two separate projectors 42A, 42B. Projector 42A is used toprovide the OTW scene 44 on the viewing screen 48 and projector 42B isused to provide the symbology 46 on the viewing screen 48 superimposedwith the OTW scene 44.

[0027] Image generation system 50 generates the images for the OTW andHUD displays which are sent to the projectors 42A and 42B. Imagegenerator 50 may be a single data processing system or even a part ofthe host computer, but most preferably comprises two image generators50A and 50B each connected with a respective projector 42A or 42B. Inthe preferred embodiment, these generators are each specialized digitalcomputer processing systems operating simultaneously in parallel. Imagegenerator 50A is used to provide the signals representative of thedesired OTW scene 44 to projector 42A and the image generator 50B isused to provide the symbology 15 for projector 42B.

[0028] The OTW scene 44 is the simulated scene viewed for the aircraftas it moves against a background in response to maneuvering signalsproduced in response to flight path signals generated by the pilot. TheOTW scene 44 may include a number of targets, including ground targets,i.e., tanks, etc., and airborne targets, such as enemy aircraft. TheseOTW scenes 44 are generated in any conventional manner typicallyprovided in a conventional flight simulator.

[0029] The symbology image generator 50B is, as noted above, linked withand responsive to tracking apparatus 50 which tracks the location of thehead and/or at least one of the eyes of the trainee. Most preferably,other sensors may be used, but the tracking apparatus 50 is an apparatususing magnetic sensors or transducers carried on the head of the pilot,as in a helmet.

[0030] The symbology image generator 50B generates symbology imageswherein the appropriate symbology is co-located with the relevantobjects, and it also preferably includes a modeling process that altersthe symbology image to conform to vignetting and occulting object(s)that would ordinarily vignette/occult the symbology in the real HUDsystem being simulated.

[0031]FIG. 5 shows a possible set of such vignetting and occultingobject(s) as would exist in a non-pupil forming HUD. The symbology imagegenerator SOB preferably has means for modeling these masks therein,which is used to modify the symbology image to simulate their presencein th esimulated HUD. Alternatively, it is also possible to provide aseparate occulting/vignetting system receiving the symbology image fromthe symbology image generator 50B and then modifying this symbologyimage based on a model of the masks or objects to be simulated, and ondata indicative of the position of the eye of the user.

[0032] In a non-pupil forming model, the object(s) being modeledpreferably consist of large opaque planar masks with appropriately sizedand shaped cutouts matching the HUD's CRT limits 70, optical pupil(s)36, and combiner 34, with the symbology drawn either beyond the CRTcutout or in such an order as to be occulted by the mask(s). Thisoccultation is a function of the pilot's eye 19 position, mostpreferably represented by the pilot's head angular position and/ororientation relative to the position of a dummy HUD 60 in the aircraftsimulator 40.

[0033] The model of the occulting of the non-pupil forming HUD producesocculting of the symbology as would occur in an optical passage as shownin FIG. 6. The section 73 of the model for the CRT mask 70 to pupil 36is generally a conic section. The section 75 from pupil 36 to combinermask 34 is a more complex form linking the two shapes of the aperturesthereof The modeling based on this particular conceptual model alsoreduces requirements of the symbology image generator 50B as to pixelfill, as compared to using multiple masks. A further reduction can beobtained by using a single mask with the appropriate shaped openingdynamically calculated as a function of pilot head position, whichopening would correspond to the instantaneous field of view, such as,e.g., area A in FIG. 7 or B in FIG. 8.

[0034] The effect of occulting determined according to this model isfurther illustrated in FIGS. 7 and 8. In FIG. 7, a field of view A ofthe overall symbology image 77 is generated and projected for the givenplacement of the eye 19 of the user. As seen in FIG. 8, when the eye iselevated slightly, a smaller, partly occulted field of view B isgenerated and projected for the pilot to see. through the beam splitter18.

[0035] Similar occluding effects occur in left and right movement.

[0036] To put it another way, in a real vehicle the actual symbologyproduced by the HUD is collimated, vignetted and occulted by the opticaland physical properties and limitations of the specific HUD design. As aconsequence, depending upon the pilot's head position, the HUD opticsmay provide to the pilot only a limited viewable portion of thesymbology, i.e., a subset of the total symbology. In order for the pilotto see more of the symbology, the pilot in an actual aircraft must movehis head, and what is seen may be analogized to what is seen through aporthole. A move to the left will provide additional viewing of thesymbology to the right, and visa versa. The same can be said for viewingup and down. Moving inward toward the HUD generally increases the amountof viewable symbology while moving away generally decreases the amountof viewable symbology.

[0037] The object(s), here mask(s), modeled by the symbology imagegenerator 50B (FIG. 4) described in connection with FIG. 5, can take onany shape and can have feathered edges (levels of transparency goingfrom opaque to transparent) as required. By creating a representation ofthe simulated HUDs physical pupils and optical limitations, thesymbology image generator produces properly vignetted and occultedsymbology. The mask(s) is modeled as appropriate to the capabilities ofthe image generator used (surface models, transparencies, texturepatterns, etc.). If the pupil shape or size changes with pilot eyeposition, then multiple pupil masks are developed, retrieved and movedas moving objects as required during the generation of the HUD symbologyimagery.

[0038] To complete the illusion for the pilot being trained in thesimulator system 40, a non-optically functional HUD (i.e., the dummy HUD60 and beamsplitter 18) are placed in the expected position in thecockpit to provide the pilot the physical “look and feel” of the HUDchassis and combiner structures.

[0039] The HUD symbology image itself is generated in the symbologyimage generator 50B and is projected onto the screen 48 with the OTWscene 44 as a real image for the pilot. The HUD symbology 46 includesthe same symbology and information content as in an actual aircraft HUD,in the expected “green” monochrome/gray scale or full color asappropriate to the aircraft HUD being simulated, albeit occulted, ifnecessary, in accordance with the position and/or angular orientation ofthe pilot's head relative to the dummy HUD chassis 60.

[0040] To simulate a collimated OTW scene 44 and HUD symbology 46, thepilot's head position, and hence eye position, is actively determinedusing a standard head-tracking device placed on the pilot's head. Thehead location is then known in all six degrees of freedom relative tothe cockpit. The determined eye position is then used to modify the OTWand symbology image generators, using determined window definitions andviewpoint location such that the displayed imagery remains correct inperspective to the pilot. With such conventional head tracking data, theOTW scene 44 and the HUD symbology 46 is moved proportionally to the eyepositional data provided by the tracking system. For example, if thepilot moves his head to the left 1″ and up 1″, the visual scene, boththe OTW scene 44 and the occulted symbology 46 of the screen 48 are alsooffset the same 1″ to the left and up 1″, producing the illusion of acollimated image (at infinity) for both OTW scene 44 and occultedsymbology 46 on the screen 48.

[0041] Referring to FIG. 4, based on the head tracking data provided bythe tracking system 50 relative to the HUD 60 for the vehicle beingsimulated, the actual field of view is calculated for the symbology inthe symbology image generator 50B. Based on these calculations, thesymbology 44 produced by the symbology image generator 50B is generatedwith any vignetting and occlusions required for the vehicle beingsimulated via the modeling of vignetting and occulting objects or masks.The vignetted and occulted symbology is thereby generated by thesymbology image generator SOB, transmitted to the symbology projector42B, and then projected onto the screen 48 superimposed with the OTWscene 46 from the OTW projector 42A.

[0042] The only symbology image projected is the viewable portion of thesymbology that is expected to be seen by the pilot as his head moves,based on the relationship of the pilot's eye and the simulatedvignetting and occulting object(s), here the mask(s) shown and describedin connection with FIG. 5, is projected onto the screen by the symbologyprojector. Consequently, the pilot can then never look “around” the HUDbeamsplitter and see the real image on the screen, because this would beoutside the modeled viewing passage 73 and 75. This arrangementtherefore provides to the pilot the illusion of a virtual image createdby an actual HUD optical system, and not as an image projected onto thescreen 48 (FIG. 4). The arrangement thus provides virtual vignetting insimulating a virtual image as viewed through the constrained size of theHUD magnifying optics using a real image as the source, i.e., thesymbology, which may be occulted as a function of pilot position,projected onto the screen by the symbology projector.

[0043] Generally speaking, the optics of HUD systems may be eitherpupil-forming or non-pupil forming systems. It should be noted that thevignetting and occulting mask(s) used in the symbology image generatormay be designed to provide the proper effects whether the optics of theHUD being simulated are pupil or non-pupil forming.

[0044] If the HUD type for the vehicle being simulated is pupil forming,then a further external pupil is created by the HUD optics and must betaken into account in the symbology image generator. In this case adynamic vignetting/occulting mask, representing the external pupil, ismodeled, in combination with other stationary masks as discussed above.This model simulates the mask dynamically, i.e., as being positioned andmodified in response to the pilot eye position to get the desiredvignetting and occulting effects. As long as the pilot keeps his eyewithin the external pupil, the HUD image is constrained only by thenormal vignetting and occulting effects described above (i.e. themodeled mask associated with the external pupil is adjusted, aperture orcutout enlarged, to be non-interfering). However, if the pilot moves hiseye outside the external pupil, the HUD image is completely blocked andthe symbology image disappears (i.e. the mask associated with theexternal pupil is adjusted, aperture or cutout reduced, to totally blockthe symbology).

[0045] The terms used herein should be read as terms of descriptionrather than of limitation, as those of skill in the art with thisspecification before them will be able to make modifications thereinwithout departing from the spirit of the invention. Other embodimentsbeyond those here discussed are within the spirit and scope of theappended claims.

What is claimed is:
 1. A vehicle simulator for a human user, said simulator comprising: tracking apparatus providing a time-dependent signal relatable to or representative of a position of an eye of the user; an image system forming a visual image viewable by the user, said image including a scene representing a simulated view outside the vehicle and head-up display symbology superposed on said scene; said image system including an image generation system for generating the scene and symbology, said scene and symbology being generated as a function of the signal provided by the tracking apparatus so that the symbology simulates optical characteristics of an actual head-up display.
 2. The simulator of claim 1, and the tracking apparatus including a apparatus supported on the head of the user.
 3. The simulator of claim 2 and the tracking apparatus including magnetic sensors, sonic sensors, or optical sensors operatively associated with the apparatus, or mechanical coupling with the apparatus on the head of the user.
 4. The simulator of claim 1 and the tracking apparatus generating a signal representing data including a position of the head of the user and an angular orientation thereof.
 5. The simulator of claim 1 and said image being projected on a projection screen.
 6. The simulation of claim 5, and said image system including a projector projecting said image on the screen.
 7. The simulator of claim 5 and said image system including first and second projectors, said first projector projecting said scene on the screen and the second projector projecting the symbology on the screen.
 8. The simulator of claim 1 and further comprising a dummy beamsplitting structure supported on the simulator, said symbology being positioned on the screen so that said symbology can be viewed by the user only through said dummy beam splitting structure.
 9. The simulator of claim 8 and said image system forming said symbology in the image only where the user can see said symbology through the dummy beamsplitting structure so that the user cannot see the symbology without looking therethrough.
 10. The simulator of claim 1 and said image generation system comprising a scene generation system and a symbology generation system.
 11. The simulator of claim 10 and said scene generation system and said symbology generation system each being an independent processor receiving a signal derived from the tracking-apparatus signal.
 12. The simulation of claim 10 and the surface being a projection screen.
 13. The simulator of claim 12 and said scene generation system being connected to a scene projector that projects the scene on the screen, and said symbology generation system being connected with a symbology projector projecting the symbology on a smaller part of the screen.
 14. The simulator of claim 1 and said image generation system simulating occulting of the symbology in a real HUD display based on the tracking apparatus signal.
 15. The simulator of claim 1 and said image generation system simulating vignetting of the symbology in a real HUD display based on the tracking apparatus signal.
 16. The simulator of claim 1 and tracking apparatus providing time-dependent data relatable to or representative of positions of two eyes of the user.
 17. The simulator of claim 1 and said image system forming the visual image on an LCD display.
 18. The simulator of claim 1 and said image system forming the visual image on a CRT display.
 19. The simulator of claim 1 and said image system collimating the scene and symbology based on the signal provided by the tracking apparatus so that the symbology and the scene stay in correct perspective with respect to movement of the head of the user.
 20. The simulator of claim 19 and said collimating of the images being at infinity.
 21. A vehicle simulator for a human user, said simulator comprising: a tracking apparatus for providing a time-dependent signal relatable to or representative of a position of an eye of the user; an image system projecting a visual image on a surface viewable by the user, said image including a scene representing a simulated view outside the vehicle and a head-up display symbology image superposed on said scene; said image system including first and second projectors, the first projector projecting the scene onto the surface, and the second projector projecting the symbology image onto the surface; and an image generation system connected with the projectors for generating the scene and symbology image, said image generation system including a symbology image generator; said symbology image generator generating said symbology image by first determining symbology for the scene and then modifying said symbology to produce the symbology image with portions thereof occulted based on the signal provided by the tracking apparatus so that the symbology simulates optical characteristics of an actual head-up display.
 22. The simulator of claim 21 and the symbology image generator generating the symbology using a model of optical characteristics including a model of a mask having a passage aperture therein.
 23. The simulator of claim 21 and the symbology image generator generating the symbology image using a model including a plurality of masks having apertures therein.
 24. The simulator of claim 23 and said model including at least one dynamic mask having an aperture, such as to model a pupil forming HUD display.
 25. The simulator of claim 21 and said symbology image generator generating the symbology image so that the symbology is collimated with objects in the scene.
 26. The simulator of claim 21 and a dummy beamsplitter structure through which the user can see a portion of the screen, said symbology image being generated and projected based on the tracking apparatus signal so that the user cannot see the symbology image around the beamsplitter structure.
 27. A method of simulating a vehicle for a user, said method comprising: determining a position of an eye of the user; projecting on a simulator screen a scene corresponding to a simulated situational environment of the vehicle; deriving a symbology image from the determination of the position of the eye of the user using a model of occulting or vignetting of an optical system of a HUD display for the vehicle so that a portion of said symbology image is occulted; and projecting said symbology image on said screen.
 28. The method of claim 27 and said deriving of the symbology image including determining an occluded symbology image using a model of an optical system using at least one mask having a central aperture therein through which the user can see.
 29. The method of claim 27 and said projecting of the scene and the symbology images being by separate projectors.
 30. The method of claim 27 and said symbology image and the scene being collimated so that objects thereof move together responsive to movement of the head of the user. 